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Introduction and Overview 



The Increasing Course Rigor Pilot Project, a joint initiative of the National Governors 
Association (NGA) Center for Best Practices and ACT, Inc., is one component of Phase Two of 
the NGA Center for Best Practices Honor States Grant Program. The project, which began in 
winter 2006 and concluded in summer 2007, was designed to improve the quality of learning 
experiences in core preparatory high school courses. The project was also a natural extension 
of ACT’s efforts to improve the quality and consistency of high school coursework. In particular, 
it is a response to empirical evidence demonstrated in the 2004 study On Course for Success: 
A Close Look at Selected High School Courses That Prepare All Students for College (ACT and 
The Education Trust) indicating the need for increased rigor and relevance in core 
preparatory courses. 

Educators and policy makers from three states (Mississippi, Oklahoma, and Pennsylvania) 
formed a unique partnership with ACT’s test developers, curriculum writers, and professional 
development specialists to accomplish a twofold goal: 

1. To investigate ACT’s approach to increasing rigor in English 10, Geometry, and Biology 
courses 

2. To identify the critical policy considerations that support increased course rigor 

The Michael and Susan Dell Foundation, The GE Foundation, State Farm Insurance, and 
ACT, Inc. provided support for this initiative. 

This report describes the results of the 17-month pilot. Both qualitative and quantitative 
findings are presented, along with implications for policy and practice. The report concludes 
with a discussion of lessons learned and recommendations for next steps. 




“Before having the opportunity to use some of the tools and 
techniques found in the ACT materials ... we looked at the 
abilities of our students as ‘limited’ ... we would give up more 
easily and accept mediocrity, which seemed to be all the 
students would give.” 

“Using the lessons and techniques as well as sharing ideas with 
the other schools in the pilot program proved beneficial.” 

“We feel that we can set the bar higher and challenge students 
to do more than the bare minimum. The materials have 
provided us with many new ways to engage students.” 

Comments from Pennsylvania English teachers 
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Project Goal, Deliverables, and Indicators of Success 



The NGA Center for Best Practices and ACT, Inc. developed the Increasing Course Rigor Pilot 
Project to achieve a specific goal and to yield four deliverables. Corresponding indicators of 
success provide a way to measure the project’s effectiveness. 

Project Goal 

To develop statewide strategies for increasing the rigor of core courses in high school that 
prepare all students for college and workforce training 

Deliverables 

1 . A professional development support system that is responsive to teacher needs 

2. Benchmark and End-of-Course Assessments to monitor student progress and course 
achievement 

3. A data-feedback model to guide intervention and assist in instructional decision making 

4. State policies to increase the rigor of core courses in high school 

Indicators of Success 

1 . Increased student readiness for college 

2. Increased student knowledge in core-course subject matter 

3. Increased teacher appreciation of course rigor and professional development support and 
increased monitoring of student progress and use of data-driven instructional interventions 

4. Improved course alignment to rigorous course objectives 

5. Improved consistency in course quality across classrooms within schools and districts 

6. Improved collaboration among teachers and school leaders to improve course quality 

7. Active state-level consideration of policies to increase the rigor of core courses in high 
school 

The project directly supported the NGA Center for Best Practices’ Action Agenda, which 
includes the following goals: 

1. Making all students prepared and proficient 

2. Redesigning the American high school 

3. Ensuring that high schools have excellent teachers and principals 

4. Holding high schools and colleges accountable for student success 

5. Streamlining/improving education governance 
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Project Participants 



Table 1 .1 : All Project Teachers 



Through a competitive application process, the 
NGA Center for Best Practices awarded 
Increasing Course Rigor Pilot Project funds to 
three states: Mississippi, Oklahoma, and 
Pennsylvania. Each state appointed a fiscal 
agent to handle funds and a state leadership 
team of policy makers and practitioners to 
provide advice and oversight. Appendix A 
presents the names and affiliations of state team 
members and state fiscal agents. 

The pilot project was implemented in eighteen 
high schools: eight in Mississippi, four in 
Oklahoma, and six in Pennsylvania. Appendix B 
displays student population characteristics for 
each school. The school names have been 
removed for confidentiality purposes. 




Ninety-eight English 10, Geometry, and Biology I 
teachers participated in the project, along with 
twenty district and building administrators. All 
ninety-eight teachers participated in at least one 
component of the pilot project. This included 
attending any of the professional development 
activities, administering the Benchmark and/or 
End-of Course Assessments, or completing one 
of the three project survey tools. Table 1 .1 shows 
a tally of the teachers by state and subject. 
Appendix C provides a more detailed listing of 
the participation level by building. 

Some teachers were more fully engaged in the 
project than others. Table 1.2 provides the 
breakdown of these sixty-three teachers, 
considered full participants for the purposes of 
this report. The full-participant teachers (and their 
students) were isolated for most of the report’s 
data analysis and observational judgments. Full 
participant teachers completed the Course 



Table 1.2: Full-Participant Teachers 
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Analysis for Rigor and Success (CARS) survey in both 2006 and 2007 and also attended at 
least one of the three major professional development workshops (Summer Institute, fall 
workshop, spring workshop). The CARS results provide both pre- and post-project curriculum 
alignment data for each teacher. The three major workshops offered vital opportunities to 
engage the research-based innovations in assessment and instructional methodology that 
ACT developed for the pilot project. The nature of the CARS survey and major workshops 
made them the logical criteria for narrowing the number of teachers. 

Professional Development Activities 

The Increasing Course Rigor Pilot Project included an array of activities, all designed to 
support participants as they worked to meet the stated goal. A time line of project activities 
appears in Appendix D. Participation in some of these activities was often limited due to the 
need for teacher time off and district scheduling conflicts. Seventy-eight teachers were able to 
attend at least one of the major workshops. The twenty who did not attend were often involved 
in the Distance Conferences or On-Site Collaboration meetings. 

Orientation Sessions 

Formal project activities began in Washington, D.C. on February 14, 2006, when NGA and ACT 
project staff oriented team leaders from the three participating states. During March and April 
2006, staff from the two agencies conducted seven in-state meetings to share information 
about the project with participating teachers and building administrators. 

Summer Institute 

The 2006 Summer Institute was the initial professional development experience for project 
participants. In late July, 114 educators (teachers, administrators, and state officials) gathered 
in St. Louis, Missouri, to address the essential questions associated with developing and 
implementing rigorous coursework. Table 2.1 shows the attendance numbers for this initial 
event. At the Summer Institute, participants examined the consistency between their current 
course objectives and ACT’s rigorous course objectives. They developed action plans to 
address existing gaps and learned how to effectively use ACT’s research-based teaching 



Table 2.1 : Summer Institute Participation 



State 


Biology 1 
Teachers 


English 10 
Teachers 


Geometry 

Teachers 


Administrators 
and Resource 
Personnel 


State 

Officials 


Mississippi 


9 


10 


10 


13 


2 


Oklahoma 


4 


7 


11 


15 


4 


Pennsylvania 


6 


7 


11 


5 


0 


Total Participants 


19 


24 


32 


33 


6 



5 




tools, which were the basis for the data-feedback 
model ACT presented. Appendix E shows a 
graphic representation of ACT’s data-feedback 
model. 

Fall Workshops 

During October and November 2006, ACT staff 
and consultants conducted a two-day professional 
development workshop in each state. Facilitators 
led a total of sixty-four teachers through focused 
discussions on four topics related to increased 
course rigor. Table 2.2 shows the teacher 
attendance for the fall workshops, which covered 
topics such as scoring constructed-response 
items, using ACT’s Template to Examine 
Assignments for Rigor and Relevance , and 
reviewing research-based strategies. The 
sessions also featured a demonstration of how to 
use the project’s discussions forum. Participants 
appreciated the issues explored, the handouts 
provided, the hands-on activities, and the 
presenters’ expertise. Session participants 
responded positively to opportunities for 
collaboration with colleagues: they enjoyed having 
a chance to interact, to share their thoughts, and 
to talk about problems that might arise in their 
classrooms. 

Spring Workshops 

ACT conducted a second round of two-day 
workshops in February and March 2007. The 
agenda for these sessions allowed for more in- 
depth discussion of student assignments, 
teaching strategies, and constructed-response 
items. Seventy-two teachers attended these 
workshops, as detailed in Table 2.3. The spring 
topics included learning to use the results of 
constructed-response items, understanding the 
components of rigorous assignments, and 
exploring research-based strategies. 



Table 2.2: Fall Workshop Participation 




Table 2.3: Spring Workshop Participation 
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Distance Conferences 



ACT conducted three rounds of distance conferences designed to support teachers’ 
implementation of the project. Each teleconference brought together all teachers of a given 
subject in the same state, along with administrators, state officials, and ACT staff. Table 2.4 
shows the teacher participation for each state. In fall 2006, seventy teachers participated in a 
teleconference focused on their successes and challenges in using ACT’s Instructional Units 
and Benchmark Assessments. The second round of teleconferences, which involved fifty 
teachers, occurred in early December 2006 and spotlighted teacher use of formative 
assessment data to inform instruction. In February 2007, the third set of teleconferences 
consisted of open discussions with ACT’s Master Teachers. Only thirty-eight teachers were 
able to take part in this teleconference. See Appendix F for a more detailed summary of the 
Distance Conferences. 



Table 2.4: Distance Conference Participation 



State 


Distance 


Distance 


Distance 


Conference 1 


Conference 2 


Conference 3 


Mississippi 


33 


20 


11 


Oklahoma 


21 


17 


15 


Pennsylvania 


16 


13 


12 


Total Teachers 


70 


50 


38 



An additional collaborative mechanism was available to teachers through seven monthly 
On-Site Collaboration meetings held at the schools and supported by ACT. A lead teacher or 
building administrator convened these meetings, which had a dual purpose: to facilitate 
teamwork that would result in increased course rigor and to begin creating a structure within 
buildings and districts to sustain these efforts beyond the life of the project. ACT and the NGA 
intended these conversations to promote refinement of practice as well as to provide support 
for implementation. Appendix F contains a detailed summary of these collaborative meetings 
with emphasis on the common appraisals (both positive and negative) shared across schools 
and subjects. 

Participants also had access to a discussions forum located on the project website (see 
Appendix G). This communication tool was used throughout the project as a means for 
teachers to share their experiences and to address questions and concerns to ACT staff and 
consulting Master Teachers. 
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Course Alignment and Teacher Practices 

At the beginning of the project, participating teachers completed three web-based 
questionnaires. The Course Analysis for Rigor and Success (CARS) survey identified gaps in 
curriculum alignment between each teacher’s courses and ACT’s objectives. The other two 
instruments yielded information about teaching practices and course structures. Table 3 
provides a brief description of the surveys, and Appendix H presents the teacher completion 
rates for each instrument. Teachers responded to two of these instruments, the CARS and the 
Course Information Questionnaire, at both the beginning and end of the project. The second 
CARS surveys enabled ACT to measure the changes in teacher alignment to rigorous 
objectives resulting from their experience in the Increasing Course Rigor Pilot Project, whereas 
the post-project Course Information Questionnaire shed light on changes in teaching practices. 



Table 3: Web-Based Surveys 



Survey 


Completion 


Description 


Course Analysis for Rigor 
and Success (CARS) 


• Spring or Fall 2006 

• Spring 2007 


Determined teacher 
alignment to ACT’s 
rigorous course objectives 


Teacher Background 
Questionnaire 


• Spring or Fall 2006 


Gathered information on 
teacher experience and 
practices 


Course Information 
Questionnaire 


• Spring or Fall 2006 

• Spring 2007 


Collected information on 
the students in each 
selected course and on the 
teacher’s instructional and 
assessment methods 



Course Analysis for Rigor and Success 

The CARS survey was designed to obtain data on objectives ACT developed as a result of the 
2004 study, On Course for Success. The survey instructed a teacher to indicate, for each ACT 
course objective listed, whether or not the objective was included in the teacher’s course. 
“Included” meant the objective was taught and, by the end of the course, students were 
expected to demonstrate proficiency. “Not included” meant the objective was not taught in the 
course, was taught in a subsequent course, or was already mastered. 

The surveys administered in spring and fall 2006 provided the foundation for evaluating the 
curriculum and methodology of each teacher. The early surveys helped frame the structure 
and goals of the project’s professional development activities, beginning with the Summer 
Institute in St. Louis. The spring 2007 surveys informed ACT’s analysis of changes in teacher 
classroom practices and curriculum objectives based on participation in the pilot project. 
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Table 4 presents the change in pre- and post -CARS results for all teachers completing both 
surveys. The percentages reflects the average change in alignment to ACT objectives from 
Fall to Spring. A more detailed analysis of teachers who demonstrated the greatest increase in 
alignment is included later in this report. 

Table 4: Percentage Change in Teacher-Reported Alignment (Fall 06 to Spring 07) 



State 


Biology 


English 10 


Geometry 


Mississippi 


2% 


9% 


5% 


Oklahoma 


2% 


14% 


-3% 


Pennsylvania 


2% 


10% 


4% 



For each of the CARS surveys, a minimum rate of 75% was used to determine whether a 
teacher (or group of teachers) was sufficiently aligned to ACT objectives. This metric reflects 
two aspects of alignment: the degree to which a teacher (or group of teachers) includes ACT 
objectives in a course and the degree of consistency — relative to course objectives 
addressed — within a group of teachers at the same building. To meet the 75% criterion, 73 out 
of 97 Biology I objectives must be taught, 68 out of 90 English 10 objectives, and 52 out of 69 
Geometry objectives. Chart 1 shows the comparison between Fall and Spring alignment to 
ACT’s objectives for each of the three subjects. For more specific building level data, see 
Appendix I. The alignment results contained in both the chart and appendix reflect the number 
of teachers in each course who reported “inclusion” of at least 75% of the ACT objectives. 




Biology I (17 Total English 10 (21 Total Geometry (25 Total 

Teachers) Teachers) Teachers) 
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Seventeen Biology teachers completed both CARS surveys, along with twenty-one English 10 
teachers, and twenty-five Geometry teachers. As the chart indicates, alignment to English 10 
and Geometry objectives increased in spring 2007, while fewer Biology teachers aligned with 
75% of the ACT objectives than at the beginning of the project. After a year of working with 
the ACT material, it was likely that some of the Biology teachers realized their understanding of 
the rigorous objectives at the beginning of the project had been incomplete. 

Instruction and Assessment Tools 

ACT Instructional Units 

Throughout the school year, teachers used material from ACT’s Instructional Units in their 
classrooms and provided feedback. See Appendix J for a complete list of the units released to 
teachers. These units were developed around high-level, college-oriented course objectives 
and emphasize flexible pedagogical styles. Six units were released for each of the three 
courses, with the last set posted to the secure website in March 2007. ACT encouraged the 
teachers to draw from these model units whenever possible, combining them with their own 
curriculum or using them as complete units. The Instructional Units provided a framework for 
the activities at the Summer Institute and the fall and spring workshops, giving teachers the 
opportunity to model the methods and classroom activities as a group. 

Each Instructional Unit was designed to culminate in a Benchmark Assessment. Early in the 
project, teachers were given concordances that matched the objectives of each unit with the 
appropriate Benchmark Assessment. They were asked to administer the assessment after 
completing each unit and to use the resultant data when evaluating the effectiveness of the 
unit methods and activities. 

End-of-Course and Benchmark Assessments 

Two major components of the Increasing Course Rigor Pilot Project were its complementary 
testing tools. The summative (End-of-Course) and formative (Benchmark) assessments were 
designed to align with the content objectives of the three subject areas. The production of 
these assessments followed the same research-based development methods ACT uses for all 
of its testing materials. 

The purpose of ACT’s End-of-Course program is to ensure that outcomes of core academic 
courses are aligned with college- and workforce-readiness standards. To that end, ACT 
conducted several research studies and surveys (e.g., On Course for Success) to determine 
the appropriate teaching objectives for individual high school courses, including English 10, 
Biology I, and Geometry. These course objectives were developed with input from curriculum 
specialists, educators, and assessment specialists. Once the course objectives were finalized, 
ACT test development staff contracted with professional educators (secondary and post- 
secondary teachers) for the development of test items. These items underwent multiple 
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content and editorial reviews by ACT staff. In spring 2006, items were field-tested in multiple 
forms. The resulting data were analyzed, and items were revised and edited as necessary. 
Operational test forms were then created from the pool of field-tested items. Those forms were 
reviewed internally by ACT staff and externally by national content and fairness review panels 
before being administered to participating project schools. 

The End-of-Course Assessments were administered from May-June 2007. Ninety-one 
teachers were sent End-of-Course materials to test their students. Of the estimated 6,700 
potential test-takers, approximately 4,200 (63%) took the End-of-Course Assessment, with the 
highest rate, 69%, in Geometry. Nearly 65% of the Biology students took the test, as did 55% 
of the English 1 0 students. The total number of students who took the End-of-Course 
Assessment in each subject and the mean of their percent-correct score is shown in Table 5.1. 
Mean scores for each building are listed in Appendix K. 



Table 5.1 : End-of-Course Assessment Results 



State 


Subject 


Student 

Count 


Mean % 
Correct 


Mississippi 


Biology 1 


879 


40% 


English 10 


384 


51% 


Geometry 


843 


34% 


Oklahoma 


Biology 1 


231 


41% 


English 10 


402 


57% 


Geometry 


401 


36% 


Pennsylvania 


Biology 1 


248 


46% 


English 10 


428 


58% 


Geometry 


381 


36% 


Total Students Tested 


Biology 1 


1358 


41% 


English 10 


1214 


55% 


Geometry 


1625 


35% 



The Benchmark Assessments were designed to give teachers formative data on student 
mastery of content and skill objectives. Eight Benchmark Assessments were developed for 
each of the three courses. Each item was aligned to a specific objective in ACT’s Increasing 
Course Rigor curriculum. The items encompassed a range of cognitive levels to enable the 
students to demonstrate their levels of mastery. The assessments were designed so that 
analysis of student performance on each individual item would assist the teacher in identifying 
areas in which an individual student, or group of students, needed additional instruction. Each 
Benchmark contained both a multiple-choice section (20-30 items) and a constructed- 
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response section (2-5 prompts). The Benchmark items required a complex understanding of 
the targeted objective and were designed to complement the rigorous course curriculum and 
facilitate innovative pedagogical practices. 

The initial goal of the project was for each teacher to administer at least four out of the eight 
Benchmarks developed for their subject. Due to many factors (planning and scoring time 
required, lack of schedule alignment, and commitments to state-mandated curriculum), the 
Benchmarks were not administered as frequently as intended. 

As Table 5.2 indicates, only twenty-nine of the ninety-eight teachers reported grades to ACT 
for at least two of the Benchmark Assessments. Twenty-one of the sixty-three full-participant 
teachers reported scores for at least two Benchmarks. Since they were given the 
corresponding first Instructional Units at the Summer Institute, a larger number of teachers 
administered the corresponding first Benchmark than any of the other seven. Forty of the sixty- 
three full-participant teachers administered Benchmark 1 . Table 5.3 includes the number of 
reported test-takers and their mean percent-correct scores for the Benchmark 1 multiple- 
choice section. 



Table 5.2: Teachers Reporting Scores 
for at least Two Benchmarks 




Table 5.3: Benchmark 1 Results (Multiple Choice) 



State 


Subject 


Student 

Count 


Mean % 
Correct 


Mississippi 


Biology 1 


412 


44% 


English 10 


267 


65% 


Geometry 


313 


60% 


Oklahoma 


Biology 1 


206 


48% 


English 10 


373 


63% 


Geometry 


366 


57% 


Pennsylvania 


Biology 1 


195 


47% 


English 10 


234 


67% 


Geometry 


122 


67% 


Total Students 
Tested 


Biology 1 


813 


46% 


English 10 


874 


65% 


Geometry 


801 


60% 
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Pre- and Post-Test Data 



As part of the project design, students in participating teachers’ courses took assessments at 
the beginning and end of the project in an attempt to gather baseline measures of college 
readiness. In fall 2006, students took the PLAN® assessment. These students then took one of 
three post-test options (PLAN, ACT® or Practice ACT®) in spring 2007, allowing ACT to 
compare the two data points to determine changes in readiness. The English scale score was 
used for the English 10 students, the Math scale score for Geometry, and the Science scale 
score for Biology. Composite scores were not used in data analysis. 

All eighteen schools administered the fall 2006 PLAN to students in project teacher classes. 
ACT then isolated the rosters of the full-participant teachers (those who had taken the first 
CARS survey and attended at least one of the three major workshops). Post-test materials 
were shipped to the project schools in spring 2007 with directions to test only the students who 
were listed on the full-participant rosters. Fifteen of the eighteen buildings were able to 
administer the PLAN, ACT, or Practice ACT during May and June 2007. It should be 
recognized that these students were also taking course-based assessments, state end-of- 
instruction tests, and ACT’s End-of-Course Assessment. It is likely that testing fatigue and 
motivation became factors affecting student performance. Because of this, ACT refined the 
test-data analysis to isolate the students who performed better than “chance” on the tests. 
These students were more likely to have put forth genuine effort when taking the three-hour 
long post-test. Table 6 shows the change in scores for all test takers who scored greater than 
chance. 



Table 6: Change in Scores from Pre-test to Post-test 



State 


Scale Score 


Student 

Count 


“>Chance” 

Count 


Mean Score 
Change 


Mississippi 


Science 


409 


303 


-.43 


English 


353 


317 


-.16 


Math 


282 


252 


+.61 


Oklahoma 


Science 


176 


135 


+.60 


English 


288 


263 


-.09 


Math 


363 


294 


+.56 


Pennsylvania 


Science 


96 


83 


+.37 


English 


252 


222 


+.24 


Math 


201 


175 


+.36 


Total Students Tested 


Science 


681 


521 


-.12 


English 


893 


802 


-.03 


Math 


846 


721 


+.45 



13 



Common Characteristics of Successful Participants 

The assessments used in this project yielded mixed results when looking at the aggregation of 
all students, thus making it difficult to interpret comprehensive findings relative to the first two 
indicators of success: 

1 . Increased student readiness for college 

2. Increased student knowledge in core-course subject matter 

However, using the test data in conjunction with other selected aspects of the project helps to 
sharpen the analysis of project outcomes and focus on areas where common characteristics of 
success become visible. The two main areas of focus used for this purpose are alignment to 
ACT’s rigorous course objectives and participation in the project’s professional development 
activities. 



Correlation Between CARS Alignment and End-of-Course Assessment Scores 



Bringing schools and teachers into alignment with ACT’s rigorous course objectives was a 
fundamental goal of the Increasing Course Rigor Pilot Project. Results of this goal are visible 
using a focused analysis of the CARS survey results and End-of-Course Assessment data. 

The rate of change in alignment with the ACT objectives from fall to spring illustrates the 
impact of that component of the project on student test performance. A measurable connection 
exists between a teacher’s change in alignment with ACT objectives and student performance 
on the End-of-Course (EOC) Assessment. Chart 2 compares the alignment rates of the 
groups with the top five and bottom five average EOC scores across all three subjects. 



70 

60 

50 

40 

30 

20 

10 

0 



Chart 2: Lowest and Highest Average End-of-Course Scores 




□ Average EOC Score 

□ % Change in Alignment 



EOC Score Groups = Students who took the 
same course at the same school. 



Average of Top 5 
EOC Score Groups 



Average of Bottom 5 
EOC Score Groups 
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Viewing these scores alongside the change in CARS alignment from Fall to Spring, a 13% 
increase in the alignment with ACT objectives is seen among those teachers whose students 
performed better on the End-of-Course Assessment. 

Conversely, students of teachers showing the highest level of correlation to ACT objectives 
performed better on the test than students in classrooms where teacher alignment actually 
decreased. Chart 3 compares the End-of-Course Assessment scores of the groups with the 
top five and bottom five average rate of change in alignment to ACT objectives. 

Chart 3: Lowest and Highest Change in Alignment Percentage 




Project Participation and Pre-/Post-Testing Results 

Additional analyses focusing on the participation of schools and teachers in professional 
development activities were selected as a reasonable approach to mining the project data. 
Using this strategy, a noticeable difference was observed in the change in the pre-test and 
post-test mean scores when the participation level of the schools and teachers was taken into 
account. School-level data were analyzed based on professional development participation 
rates, End-of-Course and Benchmark results, and CARS alignment rates. Chart 4 shows the 
difference in change in mean scores between schools that ranked in the upper third based on 
participation and alignment factors and schools that ranked in the bottom third. The results of 
the analyses revealed a difference in scores between the groups at both ends (students who 
scored less than chance were not included in the analyses). 



The students of teachers who participated at a higher rate in the professional development 
activities scored higher on the Benchmark Assessments as well as the End-of-Course 
Assessment than students of teachers less engaged and with lower participation indicators. 



15 



Chart 4: Comparison of Mean Score Change Based on Participation and Alignment 




Top Third 



1.5 



-0.9 



1.2 



□ Science 

□ English 

□ Math 



Bottom Third 



0.7 




-1.5 



-1 



-0.5 



0 



0.5 



1.5 



2 



Average Change in Pre-/Post-Test Mean Scores 
(Students performing ">Chance") 



Similarly, students in classes whose teachers had indicated a higher degree of alignment 
between their local efforts and ACT’s rigorous curriculum objectives performed slightly better 
on the post-test than students of teachers, indicating a lesser degree of alignment. 

From these analyses, it is possible to discover distinct characteristics of the teachers who were 
more aligned with ACT objectives, who participated more enthusiastically in the pilot project, 
and who ultimately improved their teaching practices. Exploring the common characteristics of 
these teachers is the key to understanding how the pilot project succeeded in achieving the 
following indicators of success: 

3. Increased teacher appreciation of course rigor and professional development support and 
increased monitoring of student progress and use of data-driven instructional interventions 

4. Improved course alignment to rigorous course objectives 

5. Improved consistency in course quality across classrooms within schools and districts 

6. Improved collaboration among teachers and school leaders to improve course quality 

A profile of successful teachers and schools emerged as a result of these comparisons. ACT 
interviews and observations, testing data, CARS survey results, and teacher feedback 
gathered from the Distance Conferences and On-Site Meetings were all used to make these 
judgments. Appendix L presents the resulting ranking system applied as a foundation for this 
evaluation. This ranking system helped illuminate three characteristics common to schools 
where students received the most benefit from teacher participation: 

• Established capacity to teach rigorous content 

• Strong instructional leadership 

• Professional communities and mentoring relationships 
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Established Capacity to Teach Rigorous Content 



In several instances, the departments and teachers who were more actively engaged in the 
work had participated in other projects that shared goals with the Increasing Course Rigor Pilot 
Project. They referenced these parallel efforts during the project’s observational and 
collaborative activities. These experiences built upon what was already present in many of the 
teachers: an established capacity to teach more rigorous content. This alignment in belief and 
practice was evident by the feedback (both positive and negative) that these teachers provided 
to ACT. The teachers had a foundation of skill and experience that enabled them to more 
actively engage the project. They implemented the instructional and assessment components 
of the project with a critical eye already focused by their established belief in the need for more 
rigor and the benefits it produces. They accepted the difficulty involved in working toward the 
project goal and were appreciative of the innovative strategies, particularly ACT’s Template to 
Examine Assignments for Rigor and Relevance. 




I think the research-based strategies were the best (part of the project). Now I can pull 
out one of my units and say, ‘What strategies can I use in this?’ It’s so nice to have 
someone give you something diat you can actually use. I will basically use the 
constructed-response items for my students when they take die (state assessments). 

-Pennsylvania Biology I teacher 



Teachers who frequently criticized the reading level of materials or the difficulty of the 
assessments were less likely to commit a high level of effort to project implementation. For 
example, these teachers were not inclined to view the presence of constructed-response items 
on the Benchmark Assessments as an important tool for instructional improvement, but instead 
saw them as an unfair strain on time and resources. Teachers with an openness to using the 
Benchmarks were appreciative of the value of formative assessments in monitoring student 
progress and adjusting their instructional planning. 




“To get some of die formative assessment strategies diat were offered— to be able to 
look at diose and say, that’s really easy and I can spot-check along and see where my 
kids are— diat has definitely helped me to develop better final assessments. ... and to 
know where I need to shift my instruction instead of just throwing it at diem and 
hoping diat it sticks.” 

- Mississippi Geometry teacher 



It became evident throughout the project that a high capacity among the educators (both in 
content knowledge and teaching practice) and an openness to new methodology are 
prerequisites for the introduction of increased rigor in the classroom. 
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Strong Instructional Leadership 

A commitment to change curriculum content and instructional practice requires leadership both 
at the building and district level. The teachers who most benefited from the opportunities the 
project provided were part of departments, buildings, and districts where leaders had made 
long-term commitments to increase rigor. Department heads, principals, and curriculum 
specialists at these schools were more likely to be directly involved in assisting individual 
teachers with implementing the project. They demonstrated an appreciation of student 
achievement centered around the project’s long-term goal of increasing student readiness for 
college and workforce training. This goal is shared by educators who recognize that change 
needs to be rooted in longitudinal curriculum development. 



At the schools that lacked strong leadership, teachers complained that their students did not 
have the content background needed before introducing ACT’s rigorous materials. However, 
teachers at schools where strong instructional leadership was present shared the project 
materials with non-project colleagues (including teachers in lower grades) in a collaborative 
effort to raise the bar for all students. Schools in the upper tier of Appendix L have 
environments focused on long-term student achievement and a desire to meet changing 
student needs and increasing accountability demands. At least two of these schools held 
planning sessions in summer 2007 to more fully incorporate the ACT materials and strategies 
into their 2007-2008 curriculum. 

Teachers also needed support from their administrators to commit time to the pilot project. 

When administrators actively supported project participation, teachers made the best use of 
the professional development sessions and were more likely to pilot multiple Instructional Units 
and Benchmark Assessments in the classroom. Teachers who were given extra time to plan 
together were better able to adapt the ACT material to their own course design. 

Professional Communities and Mentoring Relationships 

Departments that engaged the pilot project on every level and obtained some of the best 
assessment results had mentoring environments between experienced and inexperienced 
teachers. A teacher with twenty or more years of experience implementing the project alongside a 
colleague with five years or less presented an opportunity for them both to capitalize on ACT’s 
units and assessments as tools for professional growth. Within the mentoring environment, the 
younger teachers had access to both the new instructional material as well as the experience 



“It will take our students more time to adapt to die rigor of die new material. 
Incorporating this method includes raising expectations of their participation and 
performance. This can be accomplished, but in order to have it truly take root and 
develop, it will take more tiian one year.” 



-Mississippi Biology teachers 
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of the veteran teachers to assist them in applying the innovations to their classrooms. When 
the more experienced teachers were receptive to ACT’s units, assessments, and methodology, 
a collaborative enthusiasm for the project was evident to all observers. 




“We are beginning to have conversations within our professional learning communities 
about how to best use information garnered from all types of assessments. As we have 
seen the value of collaboration, we will need to continue to share strategies and build 
connections among and between departments. Our administrators are more aware of 
the need to build and sustain professional learning communities, and that has already 
contributed to a change in the structure of our in-service time.” 

-Pennsylvania English teacher 



The pilot project facilitated a level of collaboration that was new to many teachers, and their 
positive response to the potential of that type of professional environment cannot be 
overstated. Administrative support for this activity was key, and the schools that lacked active 
administrative involvement were the schools where teachers had difficulty finding time for 
internal collaboration. It was the teachers at these schools who were unable to maintain a 
minimum level of participation. The lack of collaborative support placed them on an island in 
their implementation efforts. Many of them gave up on the project rather than commit the 
individual time and effort without local encouragement or logistical support. This underscores 
the crucial importance of improved collaboration among teachers in any effort to increase 
course rigor or redesign the American high school. Administrative commitment to fostering 
collaborative environments is essential for future progress. 




“This whole project experience has created a camaraderie that is a distinct advantage in 
teaching. A comfort zone and confidence with each other encourages us in our work 
with die Geometry students. We plan to see diat new teachers in Geometry will be 
given the materials that we have decided to use and are shown what our goals are for 
die students.” 

-Oklahoma Geometry teachers 
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Lessons Learned 



At the conclusion of the project, leadership from ACT and the NGA Center for Best Practices 
collaborated on a joint list of lessons learned. They reflected on the results that could be 
discerned from data and observations, and identified the areas that would be key to successful 
future efforts to increase the rigor and relevance of core high school courses. 
Recommendations for improving the approach and methods used in accomplishing the project 
goal are listed undereach heading. 

Site Leadership/Communication 

1 . Identify a “leadership team” at each site (to consist of district/building administrators, 
curriculum staff, and classroom teachers) for constant dialogue and the mechanism for 
sustaining the work at the conclusion of ACT’s intervention. 

2. To enhance communication, appoint a site liaison who will serve as the direct contact to 
ACT and relay all information to participating teachers. This liaison would also serve on the 
leadership team. 

3. Conduct special workshops for building and district administrators, but also require them to 
attend teacher workshops. 

4. Provide teachers with guides for integrating ACT’s units with the texts they already use, or 
for using them in lieu of their usual material. 

Sequencing 

5. Early orientation meetings are critical, but they need to include explicit conversations about 
project activities and expectations. Hold the individual state meetings after the orientation 
meetings, but prior to a large-group institute. 

6. Conduct teacher observations and interviews at the beginning of the project. Devote 
adequate resources to the review and evaluation of course objectives, assessments, and 
other instructional materials that teachers submit at the beginning of the project to better 
understand participants’ practices. 

Professional Development Strategies 

7. Be more explicit about the meaning of the Course Analysis for Rigor and Success {CARS) 
results and how to align curriculum and instruction to ACT’s rigorous course objectives. 
Introduce CARS survey results to principals first, then to teachers. Address alignment 
throughout the project, not just at the beginning. Demonstrate a linkage between the CARS 
results and student test data. 
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8. Provide teachers with all instructional materials (e.g., course objectives, instructional units, 
Benchmark Assessments) at the beginning of the project. Make every effort to ensure that 
materials are relevant and timely to local instructional sequence. (The spring workshop 
based on the teachers’ classroom experiences was seen as the richest and most valuable 
professional development experience.) 

9. Require teachers to teach at least a portion of a unit then use resultant experiences 
(perhaps even videotapes of lessons) as the focus of follow-up professional development 
sessions. 

10. Do not underestimate the need to instruct teachers on how to use student answers to 
constructed-response items to inform instruction. Project teachers did not understand how 
to score these items, and they also struggled with how to use the results to guide day-to- 
day teaching. 

11. Emphasize and nurture the formation of professional communities within schools and 
districts from the onset. By the end of the project, teachers were excited about the 
collaborative relationships they formed and the benefits to their teaching. 

On-Site Modeling/Coaching/Contact Time 

12. As often as possible, send Master Teachers to sites to model best practices and to coach 
and give feedback to participating teachers. On-site professional development is more 
effective than off-site sessions. 

13. The professional development program needs to include more time with teachers. 
Communication from a distance was difficult; teachers were not strongly connected to the 
project, despite efforts (e.g., distance conferences, frequent emails). ACT staff need to be 
on-site at least twice a year to better understand participants’ situations and to establish 
strong connections with them. 
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Recommendations 



Follow Up with Project Participants 

The list of lessons learned is not exhaustive, but it highlights specific areas to improve upon 
when implementing a professional development and curriculum redesign project of this size. 
Important lessons remain to be learned from the project participants — ideas they can share 
from a new perspective now that the project is complete and they are in the middle of a new 
school year. To that end, ACT plans to contact the project participants for both the broad level 
of feedback they can offer now and to learn how their work in the Increasing Course Rigor Pilot 
Project is influencing their current curriculum and teaching practices. 

Emphasize the Importance of Alignment with Rigorous Objectives 

The rigorous course objectives presented many challenges to the project participants. Existing 
frameworks often preoccupied teachers, and if the ACT objectives did not exactly match those 
at the state or local level, it was a challenge to get people to embrace them. In some cases, 
the importance of preparing students for a particular state assessment blueprint limited teacher 
willingness to incorporate ACT objectives they did not believe were aligned. State leadership is 
needed to reassess the rigor of course objectives and facilitate innovative curriculum changes 
at the district level. 

Highlight Teaching Strategies that Work 

The teachers observed by ACT who had the most success with the project materials were 
those whose daily practices kept students constantly involved in their own learning. The 
following strategies should be promoted to increase course rigor and student achievement: 

• “Bell-to-bell” instruction 

• Instruction that is connected to prior learning 

• Instruction that is relevant to the real world 

• Instruction that incorporates probing questions, group work, and higher-level reasoning 

• Instruction that focuses on “the big picture” and that utilizes essential questions 

• Sharing the objectives and goals of daily lessons with students 

• Mandatory student notebooks 

• Incorporation of research-based strategies 

• Routine method for reporting progress to students and parents 

• Personal commitment to each and every student 
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Continue to Develop and Promote Rigorous Course Content 

The consensus among teachers was that if all of the course materials had been available at 
the beginning of the project, they would have incorporated more of the units and assessments 
into their course planning. They were committed to the goal of more rigorous content, but were 
limited in their implementation efforts by schedule and local curriculum demands. However, 
their receptiveness to the value of this challenging work should be viewed as an encouraging 
sign and spur efforts to increase course rigor nationwide. 

Continue to Promote Innovative Use of Assessments 

Research-based assessment tools are needed to evaluate whether instructional methods are 
helping students develop critical learning skills and master rigorous content objectives. The 
pilot project provided teachers with a variety of assessments (Benchmark Assessments, End- 
of-Course Assessments, and assessments embedded in the Instructional Units). Teachers 
were encouraged to use these tools in new ways. More effort should be made to emphasize 
the value of using different types of assessments as well as analyzing assessment data to 
modify instruction on a regular basis. While this project did not provide definitive support for the 
use of End-of-Course assessments, there are already many established reasons for continued 
development in that area. The correlation between End-of Course performance and an 
increase in alignment to rigorous objectives is encouraging. 

Emphasize the Need for Teacher Collaboration 

The most resounding lesson drawn from this pilot project is the understanding that when 
teachers are encouraged to form professional learning communities (and given the time and 
resources to do so) the benefits to students will be significant. Collaboration within a 
department improves teacher relationships and strengthens curriculum. Collaboration across 
departments helps teachers know their students better and coordinate activities that enhance 
overall student learning. Teacher collaboration is a crucial part of efforts to redesign 
curriculum, to develop innovative teaching strategies, and to address the needs of students in 
ways that best enable them to learn. 
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Appendix A: 

State Team Members 



Mississippi 

Rico Buckhaulter 

CSR Coordinator 


Norman Session 

Principal 

Pisgah High School 
Sandhill, MS 


Office of Innovative Support 
Mississippi Department of Education 
Jackson, MS 


Stanley Shows 

Richland High School 
Richland, MS 


George Gilreath 

Principal 

Brandon High School 
Brandon, MS 

Kristopher Kaase 

Associate Superintendent 


Mariella Simons 

Director of Instruction 
Office of Curriculum and Instruction 
Rankin County School District 
Brandon, MS 


Office of Academic Education 
Mississippi Department of Education 
Jackson, MS 


Kevin Vance 

Assistant Principal 
Pisgah High School 
Sandhill, MS 


William Lenington 

Principal 

McLaurin High School 


Oklahoma 


Florence, MS 

Tony Martin 

Principal 

Florence High School 


Rita Beisel 

Assistant Principal 
Altus High School 
Altus, OK 


Florence, MS 

Jean Massey 

Principal 

Northwest Rankin High School 


Cindy Brown 

Director of Student Preparation 

Oklahoma State Regents for Higher Education 

Oklahoma City, OK 


Flowood, MS 

Scott Parham 

Principal 

Puckett High School 


Nancy Evans 

Director of Curriculum 
Altus Board of Education 
Altus, OK 


Puckett, MS 

Shane Sanders 

Principal 

Pelahatchie High School 


Hank Harris 

Interim Superintendent 
Poteau Public School District 
Poteau, OK 


Pelahatchie, MS 

Undray Scott 

Assistant Principal 
Brandon High School 
Brandon, MS 


Cindy Koss 

Assistant Superintendent 
Office of Standards and Curriculum 
Oklahoma State Department of Education 
Oklahoma City, OK 
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Appendix A (cont.) 



Johnny Morrow 

GEAR UP Trainer/Coordinator 

Oklahoma State Regents for Higher Education 

Oklahoma City, OK 

Susan Powell 

Assessment Coordinator 
Ponca City School District 
Ponca City, OK 

Susan Savage 

Secretary of State of Oklahoma 
Oklahoma City, OK 

John Spencer 

Principal 

Poteau High School 
Poteau, OK 

Brian Staples 

Principal 

Northeast Academy High School 
Oklahoma City, OK 

Susan M. Staples 

Director 

Career and Technical Education 
Secondary Guidance Services 
Oklahoma City Public Schools 
Oklahoma City, OK 

Pennsylvania 

Tammy Baumann 

Math Facilitator 
East High School 
Erie, PA 

Rosemary Browne 

The Foundation for Enhancing Communities 
Harrisburg, PA 

Ken Bussard 

Principal 

Central Cambria High School 
Ebensburg, PA 



Rebecca Erb 

Principal 

Tyrone Area High School 
Tyrone, PA 

Susan Makosy 

Superintendent 

Central Cambria School District 
Ebensburg, PA 

Parker Martin 

Chief of the Division of Middle and 
Secondary Education 
Harrisburg, PA 

Glenn Thompson 

Assistant Superintendent 
Bedford Area School District 
Bedford, PA 

Sharon Tucker 

Research and Strategic Planning Specialist 
Office of Elementary and Secondary Education 
Pennsylvania Department of Education 
Harrisburg, PA 

Ed Vollbrecht 

Special Assistant to the Secretary 
Acting Bureau Director, Teaching and 
Learning Support 

Pennsylvania Department of Education 
Harrisburg, PA 



Patrick Crawford 

Superintendent 

Bedford Area School District 

Bedford, PA 
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Appendix B: 

Student Population Characteristics 



High School 


Total 

Enrollment* 


Student/ 

Teacher 

Ratio 


Ethnic 

Minority 


Free/ 

Reduced 

Lunch 


A1 


1,215 


15.4 


20% 


22% 


A2 


627 


15.4 


15% 


28% 


A3 


354 


11.1 


NA 


NA 


A4 


1,262 


17.2 


21% 


17% 


A5 


236 


12.2 


NA 


NA 


B1 


171 


10.4 


NA 


NA 


B2 


210 


12.6 


NA 


NA 


B3 


414 


13.1 


20% 


39% 


B4 


820 


14.6 


40% 


42% 


B5 


480 


15.6 


84% 


48% 


Cl 


1,634 


15.3 


26% 


47% 


C2 


617 


15.5 


30% 


50% 


C3 


638 


19.0 


4% 


19% 


C4 


685 


18.3 


2% 


22% 


C5 


1,193 


12.4 


39% 


68% 


D1 


1,056 


17.5 


64% 


66% 


D2 


813 


17.8 


46% 


58% 


D3 


651 


15.6 


2% 


61% 



"Enrollment figures based on 2004-2005 data from The National Center for Education Statistics. NA indicates 
NCES data reflected grades prior to 9-12, so data specific to high school years cannot be determined. 
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Appendix C: 

Teacher Participation Details 
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Appendix D: 

Project Activity Time Line 





Professional 

Development 


Survey 

Research 


Assessment 


Project 
Direction and 
Evaluation 


February 14, 2006 








Kick-Off meeting 
with state team 
leaders, NGA 
Center for Best 
Practices staff, and 
ACT held in 
Washington, D.C. 


March-April 2006 


In-state orientation 
meetings for 
participating teachers 
and administrators 








Spring (or Fall) 
2006 




Teachers took 
1st CARS 
Survey 






May-June 2006 






End-of-Course item 
tryout 




July 24-27 2006 


Summer Institute held in 
St. Louis, Missouri 








August- 
September 2006 


Discussions Forum 
created for project 
collaboration purposes 




Electronic 
Gradebook sent to 
Teachers 




September 2006 


• Distance 
Conference #1 

• Posted 1 st set of 
Instructional Units 


Administration of 

Teacher 

Background 

Questionnaire 

and Course 

Information 

Questionnaire 


Posted 1 st set of 

Benchmark 

Assessments 




October- 
November 2006 


• On-site 
Collaboration 
Meetings begin 

• Held three in-state 
professional 
development 
workshops 




Students took 
PLAN to measure 
initial college- 
readiness 




November 2006 


Posted 2nd set of 
Instructional Units 




Posted 2nd set of 

Benchmark 

Assessments 




December 2006 


Distance 
Conference #2 
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Appendix D (cont.) 





Professional 

Development 


Survey 

Research 


Assessment 


Project Direc 
tion 
and 

Evaluation 


January 2007 


Posted 3rd set of 
Instructional Units 




Posted 3rd set of 

Benchmark 

Assessments 


Reported 
baseline data 
and project 
summary to 
NGA Center for 
Best Practices 


February 2007 


Distance 
Conference #3 








February-March 2007 


• Held three in-state 
professional 
development 
workshops 

• Conducted on-site 
observations and 
interviews of 
project teachers 








March 2007 


Posted final set of 
Instructional Units 




Posted final set of 

Benchmark 

Assessments 




April-May 2007 




• Teachers 
completed 2nd 
CARS Survey 

• Teachers 
completed 2nd 
Course 
Information 
Questionnaire 






May-June 2007 






• Administered 
End-of-Course 
Assessments 

• Administered 
PLAN, ACT, or 
Practice ACT 
(college- 
readiness 
post-test) 




June-August 2007 


Observational data is 
coded and analyzed 


Project survey 
results are 
analyzed 


Assessment data is 
collected and 
analyzed 


Prepared Final 
Report 
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Appendix E: 

ACT’s Data-Feedback Model 
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Appendix F: 

Distance Conferences and On-Site Collaboration Meetings 



Distance Conferences 

Distance Conference #1 : September 2006 — Participants conveyed specific insight and criticism 
regarding their building-level implementation of the pilot project. Teachers were enthusiastic about their 
early use of the ACT Instructional Units and described their experience with piloting them in the 
classroom. During this discussion, ACT staff strongly encouraged teachers to experiment with the 
material and be creative in integrating it into their lesson plans. This practice was advocated throughout 
the project, and the teachers who followed this advice reported positive results. One issue that 
originated in this conference was the perception that the reading level of ACT’s materials was too 
challenging for the 9th and 10th graders in many of the classrooms. While teachers recognized that 
raising the reading level of students was important, they were concerned that it hindered their efforts to 
teach the content objectives in Geometry and Biology. It was during this conference that it first became 
apparent that teachers would have difficulties administering the Benchmark Assessments. Concern was 
expressed over the length of the tests, the inclusion of constructed-response items, and most 
importantly, the order in which ACT released the Benchmarks. The release order did not always conform 
with the order teachers taught the objectives, and this ultimately hindered a more comprehensive 
administration of the Benchmark Assessments. 

Distance Conference #2: December 2006 — These teleconferences centered on the results of early 
administrations of the Benchmark tests and approaches to different types of assessments. The teachers 
found the Benchmarks to be useful tools for targeting the specific needs of students. The tests were 
helpful in identifying students who required individual help in content learning as well as assistance in 
developing reading comprehension skills. The rigor of the questions, although making the tests more 
difficult overall, actually resulted in some students feeling greater accomplishment when they answered 
the questions correctly, as opposed to doing very well on an easier test. The teachers agreed that the 
constructed-response items presented students with the most difficulty, but the items were also an 
excellent opportunity for instructional reinforcement. Some teachers had students review their responses 
with the scoring rubrics to see what type of response the question was looking for and how their 
responses differed. Teachers discussed the value of various assessment tools in achieving instructional 
objectives. They shared examples from their experiences in piloting the embedded assessments within 
the ACT model units, including successful styles of assessments they had used in the past. 

Distance Conference #3: February 2007 — The third distance conference built on the relationships 
begun at the Summer Institute and the fall and spring workshops and gave the teachers a chance to 
discuss their project experiences with one of ACT’s Master Teachers. Each teleconference involved a 
discussion of student motivation, specifically the methods used to engage students who had difficulty 
adjusting to the increased rigor of the curriculum. The Master Teachers and the project participants 
discussed ideas to address these situations, along with other content-specific areas of concern. A 
thought-provoking discussion during an English 10 teleconference centered on the difficulties faced 
when teaching racially and ethnically themed literature in primarily homogeneous classrooms. 

On-Site Collaboration Meetings 

October 2006: Successes and Challenges — After over a month of piloting the material in their 
classrooms, the teachers and administrators met to discuss their impressions of the project. Many 
teachers mentioned the difficulty their regular level (non-honors, non-accelerated) students had 
adjusting to the rigorous content. It was also difficult for the teachers to adjust to using the rigorous 
material in those classes. However, the benefits of introducing rigor into those sections were clear to 
them: interest generated by the unit activities, use of higher-order thinking skills and questioning 
strategies to develop an “academic mentality.” English teachers found that developing annotation skills 
was particularly valuable. Geometry teachers liked the incorporation of logic and deduction while 
introducing proofs. Biology teachers appreciated the goal of teaching students to differentiate between 
theoretical and empirical concepts. The time involved in implementing the project was a universal topic 
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Appendix F (cont.) 



of concern. The material (both instructional and assessment) was in many ways drastically different from 
current practices, and the time the teachers had to prepare was insufficient for incorporating the material 
into their instruction. Participants found the goals and content of ACT’s materials very challenging. This 
reality underscored the need for preparation time and increased teacher collaboration — an area that 
would be emphasized as the project moved forward. 

November 2006: Template for Evaluating Student Assignments — The use and value of the 
professional development materials from the fall workshops were the focus of the November meetings. 
Teachers agreed that ACT’s Template to Examine Assignments for Rigor and Relevance helped pilot 
new ways of presenting material and made them aware of new approaches to instruction. It helped them 
to spot weaknesses in their methods and forced them to create more rigorous lessons and activities. 
They again expressed need for more collaboration time to make the most of the material. Teachers felt 
there would be more benefit if they were able to review each others’ assignments using the template. It 
was good to use individually, but the template would be more valuable if used in a group. 

December 2006: Value of Assessments — The project’s formative assessments (both embedded and 
Benchmarks) were the focus of the third on-site meeting. Teachers discussed how assessments inform 
changes in both group and individual instruction. The rigorous Benchmark Assessments, while often 
frustrating to students and teachers, were seen as providing valuable specific information on what key 
concepts and reasoning strategies the students were not learning. This supports the main concept 
behind the data-feedback model: if a class or group is not retaining knowledge or applying new skills, 
those areas need to be readdressed and instructional changes contemplated. Many Biology and 
Geometry teachers cited their inexperience in using constructed-response items like those in the 
Benchmarks. These items gave teachers the opportunity to help students view learning from different 
perspectives. Teachers taught students how to score their constructed-response items as a means to 
develop new writing and analytical skills. Another important goal they emphasized was a commitment to 
using a variety of assessment types (tests, quizzes, embedded exercises). 

January 2007: Research-Based Teaching Strategies — During the fourth on-site meeting, the 
participants discussed the various research-based teaching strategies presented by ACT, particularly 
those contained in the Educator’s Toolbox offered at the fall workshops. Among the strategies and 
activities the participants found most fruitful were reflective questioning, annotation, Cornell note-taking, 
KWL charts, designing exam questions, and fishbowl discussions. They were also enthusiastic about 
using technology innovations such as whiteboards and classroom “clickers.” However, not all teachers 
had completely positive experiences. One teacher attempted a group exercise but was unable to keep 
the students from falling into predictable patterns of either dominating or observing. These experiences 
reinforced the need for teachers to be aware of their classroom dynamics and to target activities they 
know will work versus what may be difficult for a particular group of students. Teachers also expressed 
concerns about the literature used in ACT’s English units. Certain racial and ethnic themes were not 
viewed as a good fit for every classroom. In diverse classroom environments, there existed the 
possibility that some students may find themselves in uncomfortable situations. In homogeneous 
classrooms, students may lack a personal, tangible connection to the literature being presented, and 
some teachers felt this introduced obstacles to achieving key instructional goals. Overall, the 
participants were excited and impressed by all the options in the toolbox. However, they stressed a 
desire for more modeling and training in how to employ these techniques in their classrooms. 

February 2007: Changes in Beliefs about Teaching — Many teachers responded somewhat 
defensively to this topic. Some pointed out that ACT’s materials were not in their final form at the start of 
the project; therefore, it was a difficult and artificial task to pilot as many of them as possible. One school 
labeled its focus as content driven and ACT’s as concept driven. Other teachers maintained they were 
already practicing rigorous teaching techniques, especially those who were preparing students for state 
tests in their subject areas. However, teachers in Pennsylvania and at Northwest Rankin High School in 
Mississippi acknowledged that the pilot project forced them to view their teaching practices from different 
perspectives, and by doing so, expanded the goals and content of their courses. One school saw this as 
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Appendix F (cont.) 



especially useful with those non-honors students who are prone to do the minimum and for whom 
teachers are reluctant to raise the performance bar. 

March 2007: Changes in Beliefs about Course Rigor — Several participants changed their perception 
of rigor as a result of this pilot project. Teachers who operated under the assumption that rigor dealt only 
with the difficulty of the course content learned that this is a narrow understanding. They discovered that 
increasing course rigor also involves the implementation of new strategies to develop and enhance 
active student participation in their own learning. But this discovery came with the realization that much 
more planning and collaboration are necessary to effectively raise rigor across classrooms and 
departments. This requires more preparation time with comprehensive planning. It also requires 
understanding the makeup of each class, especially those with IEP or learning support students, and 
targeting the rigorous instruction accordingly. Support from administrators is necessary, as is a 
commitment to apply these strategies in a longitudinal manner. In a rigorous course (especially 
mathematics), students need to refer to prior objectives and to apply prior knowledge. One teacher 
emphasized that course rigor is not completely dependent on the teacher or content; it also requires (at 
the same time it seeks to nurture) students to explore subjects in depth and take ownership of their 
learning. 

April 2007: Applying the Strategies Learned during the Increasing Course Rigor Pilot Project — 

The teachers’ final discussion evaluated their experiences with the project and discussed how they will 
apply what they have learned. A common frustration centered around two linked issues: the lack of time 
(both before and during the project) and the staggered release of materials. Ideally, the teachers would 
have received a panoramic view of the scope and sequence of the project that included access to all 
materials. The orientation meetings held at the buildings in spring 2006 were a basic introduction, but 
they did not present any of the actual materials and assessments the teachers were to use during the 
2006-2007 academic year. They expressed enthusiasm about the introduction of these materials during 
the Summer Institute, but the timing did not allow them to plan integration of ACT’s materials into their 
courses. It was also difficult to incorporate material that was released in intervals throughout the year. In 
Biology and Geometry, there were significant differences in the sequencing of ACT’s objectives and the 
order that many of the teachers cover content. This limited the teachers’ ability to adequately pilot ACT’s 
units and assessments. 

Even though such issues affected project implementation during the pilot year, the teachers were 
excited to more fully utilize ACT’s rigorous material in their planning for 2007-2008. This summer, 

Ponca City High School teachers were paid to collaborate on a plan for integrating as much of the 
material as possible into their courses next year. Teachers lauded the strategies presented in the 
toolbox and template, and they expressed genuine excitement at the opportunity to use those practices 
in different courses with new groups of students, as well as sharing them with other teachers and 
departments. One obstacle to increasing the rigor in all classrooms that many teachers face is requiring 
more from their “lower-level” classes, but teachers recognized the benefits of working toward this goal, 
for both the students and themselves. 
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Appendix G: 

Project Web Resources 



The home page listed the 
project goal, deliverables, and 
indicators of success. 



3 NGA/ACT Pilot Project: Increasing Course Rigor - Microsoft Internet Explorer 



File Edit View Favorites Xools Help 



File E< 
I v-< Back 
Address f" 



J j 4) ^Search _£J Favorites Media ^ . 
Address | ] http:/7 www.act.org/ngapilot/ 



I W best 

Increasing Course Rigor 



NGA Center lor 

BEST PRACTICES 



Deliverable 




► Professional-development support 
system for teachers 

► Benchmark and end-of-course 
assessments 

► Data-feedback model to assist in 
instructional decision making 

► State policies to increase course rigor 



ACT 



3 NGA/ACT Pilot Project: Instructional Units and Benchmark Assessments - Microsoft Internet 



File Edit View Favorites look Help 

J J ^Search _£| Favorites Media ^ j 



Address | ] http://www.act.org/ngapilot/secure/ 




Home 

Resource Materials 



An Increasing Course Rigor 



Upcoming Activities 






On-Site Collahoiation 
Meetings 




• Test Administration Manual 


Discussions Foium 





Contact Information 






ACT 



E0C Administration Manuals 



Instructional Units and Benchmark Assessments 



English 10 



Benchmark Assessment One 
Benchmark One Information 

Benchmark Assessment Two 
Benchmark Two Information 

Benchmark Assessment Three 
Benchmark Three Information 



Instructional Unit 1 
Instructional Unit 2 



Educators and ACT staff 
communicated throughout the 
project using the discussions 
forum. 



Instructional Units and 
. Benchmark Assessments were 
available on a secure site. 



3 NGA/ACT Increasing Course Rigor Pilot Project :: 


/iew Forum - English 10 - Mict 


soft Internet 


File Edit View Favorites look Help 




\J-> Back » ^ J J ^Search i I Favorites jJj 1 Media fe 


3-11 


ft 



m 



NGA/ACT Increasing Course Rigor 



ncreasing Course Rigor 

English 10 

Moderators: Site Moderator . Caine Thomas 
Users browsing this forum: None 

(j^l new topic) NGA/ACT Increasing Course Rigor Pilot Project Forum Index -> English 10 



® FAQ 19 ) Search 0 Register 
IS Profile Ifil Log in 



Replies Auth 



(3) Revisions to Siddhartha, Speeches, and Existentialism? 
( 3 ) Year end activities 

( 3 ) Sports Illustrated article and Shakespeare video 
Benchmark 2 

( 3 ) Talking Race in the Classroom by Jane Bolgatz 
( 3 ) Unit 3 Essay Test/ PLAN practice question 
(§) HELP NEEDED AGAIN 
@ HELP Benchmark 2 CR 



Cvnthia Miller 
Caine Thomas 
Cvnthia Miller 
PHS1 

Cvnthia Miller 

PHS1 

Mandv 

Herrington 

Mandv 

Herrington 
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Appendix H: 

Number of Teachers Responding to Surveys 
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Appendix I: 

CARS Alignment Results 
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Appendix J: 

ACT Instructional Units Released to Project Participants 



English 10 

Introduction to English 10: The First Days of School 1 1 Class Periods 

Students will be introduced to several concepts that will be important for their entire year's schoolwork, 
and they will begin a conversation about the value of reading and writing. Students will become aware of 
and learn to use metacognitive strategies when reading difficult works; they will begin to learn the skill of 
annotation. Students will also increase their vocabularies by learning the Greek and Latin derivatives of 
everyday words. Finally, students will begin learning how to write, ask, and answer literal, interpretive, 
and beyond-the-text questions about texts they read. Overall, students will begin thinking about 
themselves as readers and writers. 

Where Do I Fit In? Exploring Identity and Culture Through Literature 20 Class Periods 

Students will explore the concepts of identity and culture through reading poetry and three nonfiction 
selections. Students will gain a deeper understanding of the concepts of identity and culture, which will 
be useful to them as they read and interpret texts throughout the remainder of the course. Students will 
analyze poetic structures and devices and follow thematic links within a work and across works covered 
in the unit. 

Warriors Don’t Cry: Explorations of Culture. Identity, and History 25 Class Periods 

Students will continue their exploration of the intersection between identity and culture by reading a 
variety of nonfiction works centered around the civil rights movement of the 1950s and 1960s; an 
autobiography, excerpts from an essay, an executive summary, and a partial history of the 
desegregation of Little Rock Central High School in 1957. Students will also discuss themes of history, 
culture, and identity as they read one short story. Through reading, role playing, and viewing 
photographs and a documentary film, students will investigate aspects of the desegregation of public 
schools after the landmark education decision, Brown v. Board of Education (1954). Students will also 
write an essay about culture, heritage, and identity, in which they will describe an object, event, or story 
of importance to them. 

I Need a Hero: Joseph Campbell and Siddhartha 22 Class Periods 

Students will learn about and work with Joseph Campbell's concept of the hero's journey. They will 
practice applying the structure of the hero's journey to stories and films with which they are familiar, to 
the short story "The Man to Send Rain Clouds," and to the complex novel Siddhartha. Students will 
develop the ability to identify similarities and differences in various pieces of literature. In a final essay, 
students will compare two characters’ journeys. 

Existentialism: Questions Without Answers 15 Class Periods 

After practicing active reading and note taking on a dense critical essay about existentialism, students 
will use close reading techniques to construct deeper meanings of two short stories, a poem, and an 
excerpt from an autobiography. To gain insight into both the essay and the other literature, they will 
connect the existentialist principles described in the essay to the texts. Meanwhile, students will write a 
three-page informational essay that explains the basic concepts of existentialism. In this paper, students 
will focus more on communicating to an audience than they have in previous essays. 

Analysis of Speeches 13 Class Periods 

In this unit, students will learn about the art of persuasion as they analyze and interpret five persuasive 
speeches and write one letter to the editor. Students will also learn to identify logical fallacies and to 
distinguish fact from opinion. They will present their interpretations of modern speeches at the end of the 
unit. 
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Geometry 

Beyond a Shadow of a Doubt: Logic and Proof 9 Class Periods 

Logic can be defined as the science of reasoning. In high school geometry, students focus on producing 
and presenting logical arguments to explain and justify conjectures about geometric figures and real- 
world experiences. This unit begins with an understanding of conditional and biconditional statements 
that leads to both inductive and deductive reasoning. Students will begin to learn formal proof 
techniques using both algebraic and geometric ideas. Students will refine these techniques throughout 
the course. 

What’s So Special About Angle Pairs? 10 Class Periods 

This unit introduces students to the special pairs of angles formed by two coplanar lines and a 
transversal. Students will solve problems and complete proofs involving congruent and supplementary 
angle pairs formed by parallel lines. They will also use various angle pairs to prove that two lines are 
parallel. 

Congruent Triangles: Shortcuts 7 Class Periods 

To explore congruence and the properties of congruent geometric figures, students will discover five 
methods for proving that triangles are congruent through an inductive approach. Then, using all five 
methods, along with previously studied properties of triangles, students will prove that triangles and 
parts of triangles are congruent. Proving that triangles are congruent allows students to practice their 
deductive reasoning skills and to communicate mathematically while using the properties of triangles. 

What’s Your Angle on Polygons and Quadrilaterals? 10 Class Periods 

The following activities are designed to help students generate and apply formulas to find the measures 
of the exterior and interior angles of convex polygons and to explore the properties of special 
quadrilaterals. Students will practice solving both algebraic equations and real-life problems and 
completing proofs. 

What’s So Right About Right Triangles? 6 Class Periods 

Using the Pythagorean Theorem and its converse, students will investigate how to solve mathematical 
and real-world problems involving right triangles, Pythagorean triples, geometric mean, and the 
relationships among the sides of special right triangles. 

Going in Circles 9 Class Periods 

Students will investigate the relationships between angle measures, intercepted arc measures, and 
segment lengths formed by lines and line segments associated with circles. The equation of a circle will 
be introduced, and students will solve problems involving inscribed and circumscribed polygons. 
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Biology 

Demystifying the Nature of Science 7 Class Periods 

Students will examine the characteristics that define science as a unique way of study. 

The Five-Second Rule: A Rule to Live by or a Myth to Bust 10 Class Periods 

To reach the goals of the National Science Education Standards, students must be encouraged to 
investigate the natural world and be taught methods of scientific inquiry. This unit is designed to engage 
students in developing these important skills. It is intended to introduce students to the study of biology 
through scientific inquiry. 

Beaks. Beans, and M&M’s: A Study of Natural Selection 8 Class Periods 

This unit will help students investigate the components of natural selection through the process of 
scientific inquiry. 

Atomically Correct 10 Class Periods 

Students will use models to learn about the subatomic particles that compose atoms and how to 
distinguish between electrically neutral atoms and their ions. Then, students will determine how the 
valence electrons of atoms are involved in the formation of ionic and covalent bonds. Finally, they will 
investigate the behavior of ionic and covalent compounds in solution and the effect of pH on enzyme 
action. 

The Organization of the Biosphere 1 1 Class Periods 

This unit will help students define ecology and develop an understanding of the organization of the 
biosphere. Students will analyze different types of communities and study the effect of interactions within 
ecosystems and the role of such interactions in the process of succession. 

Mendel’s Peas: A Study of Mendelian Genetics 1 1 Class Periods 

The mechanism through which traits are passed down from generation to generation confounded 
biologists, as well as farmers, philosophers, and parents, for many years. Gregor Mendel, an Austrian 
monk and biologist, developed a model of inheritance that predicted how traits were passed from parent 
to offspring in most organisms. In this unit, students will investigate the science behind Mendel’s 
conclusions and replicate his results. 
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Appendix K: 

End-of-Course Assessment Mean Scores 



School 


Subject 


Student 

Count 


Mean 


A1 


Biology 1 


293 


34.74 


A2 


Biology 1 


54 


37.08 


A4 


Biology 1 


294 


42.21 


A5 


Biology 1 


30 


46.89 


B2 


Biology 1 


49 


56.71 


B3 


Biology 1 


159 


40.29 


B4 


Biology 1 


60 


29.34 


B5 


Biology 1 


99 


50.49 


C2 


Biology 1 


72 


38.89 


C5 


Biology 1 


116 


42.21 


D2 


Biology 1 


55 


54.59 


D3 


Biology 1 


74 


43.69 










A1 


English 10 


101 


47.78 


A2 


English 10 


43 


54.93 


A4 


English 10 


81 


58.38 


A5 


English 10 


13 


28.74 


B2 


English 10 


47 


53.22 


B3 


English 10 


99 


46.89 


B4 


English 10 


103 


57.76 


B5 


English 10 


110 


66.42 


Cl 


English 10 


44 


67.37 


C2 


English 10 


145 


46.66 


C4 


English 10 


119 


62.85 


D2 


English 10 


120 


54.58 


D3 


English 10 


81 


53.07 










A1 


Geometry 


194 


33.89 


A2 


Geometry 


117 


41.97 


A3 


Geometry 


83 


29.20 


A4 


Geometry 


170 


35.28 


A5 


Geometry 


20 


17.63 


B2 


Geometry 


25 


25.76 


B3 


Geometry 


57 


35.18 


B4 


Geometry 


73 


28.37 


B5 


Geometry 


43 


42.96 


Cl 


Geometry 


219 


38.17 


C2 


Geometry 


66 


31.22 


C3 


Geometry 


86 


28.73 


C4 


Geometry 


33 


34.32 


D1 


Geometry 


13 


32.79 


D2 


Geometry 


47 


36.59 


D3 


Geometry 


80 


44.18 
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Appendix L: 

Rankings Based on Participation Levels and Testing Data 



These tables show which groups of teachers participated at higher rates in project activities, 
whose students performed relatively better on the assessments, and whose curriculum was 
aligned at a higher rate to ACT course objectives. The departments and teachers in the upper 
tier of these rankings correspond to observational judgments of which teachers were more 
committed to the project goal and which will most likely engage in continued efforts to increase 
course rigor. The tables represent the average of rankings (with 1 being the highest and 20 
assigned to null values) of each subject area based on data for the following: 



1. Fall 2006 Pre-Test Mean Scores 

2. Spring 2007 Post-Test Mean Scores 

3. Change in Pre- Post-Test Mean based on 
“>chance” scores 



4. Professional Development Activities per 
Teacher 

5. Benchmark 1 Mean Scores 

6. End-of-Course Assessment Mean Scores 

7. Spring 2007 CARS Alignment Rates 



Only schools with subject teachers that had data for at least one of the above activities are 
presented in these tables. 



Biology I English 10 Geometry 



School 


Average 

Rank 


B5 


2.6 


D3 


4.6 


A4 


5.0 


A1 


6.0 


B4 


6.0 


C4 


7.1 


C2 


7.7 


B2 


8.4 


A2 


8.6 


A5 


9.6 


B3 


9.7 


C5 


11.0 


D2 


15.3 


B1 


18.3 



School 


Average 

Rank 


C4 


2.4 


A4 


3.9 


B5 


4.1 


D3 


5.7 


B4 


6.1 


Cl 


6.6 


A2 


7.0 


A1 


7.1 


C2 


7.7 


B3 


9.3 


A5 


9.7 


C3 


10.0 


D2 


15.1 


B2 


16.7 


B1 


18.9 



School 


Average 

Rank 


D3 


2.4 


A4 


3.9 


B5 


5.7 


B4 


6.0 


A1 


6.1 


Cl 


6.6 


B3 


6.9 


A3 


7.0 


D2 


7.6 


C4 


8.6 


C2 


8.7 


C3 


10.0 


B2 


10.9 


A5 


12.1 


D1 


15.0 


A2 


15.9 


B1 


18.4 
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